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Abstract 


A  method  of  evaluating  reliability  of  large  scale  rocket  engines  during 
a  Research  and  Development  Program  prior  to  missile  flight  tests  is  presented. 
Reliability  estimates  are  obtained  even  though  the  configuration  of  the  engine 
is  undergoing  change  and  the  objectives  of  test  firings  vary.  Each  engine  is 
apportioned  into  Principal  Subsystems  which  are  screened  for  their  degree  of 
representation  of  the  final  configuration.  The  intention  of  each  test  firing 
is  determined  prior  to  the  test  and  the  behavior  of  the  Principal  Subsystem 
when  tested  within  the  engine  environment  is  classified  according  to  pre-specified 
ground  rules  as  a  Success,  Failure  or  Exclusion.  These  results  are  then, 
statistically  combined  to  give  an  estimate  of  engine  reliability. 
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Foreword 

The  concepts  used  and  the  approach  taken  to  solve  the  particular  problem  dis¬ 
cussed  in  this  paper  may  be  generalized  to  apply  and  be  utilized  for  systems  in 
which  the  following  situations  occur. 

1)  It  is  desired  to  evaluate  the  reliability  of  the  system  against  its  end  use. 

2)  The  system  is  in  a  state  of  continuous  development  towards  an  end  configura¬ 
tion. 

3)  The  intentions  of  the  tests  vary  as  the  system  evolves  and  there  are  nc  or 
few  tests  specifically  for  reliability  evaluation. 

k)  There  is  a  limited  number  of  systems  available  for  testing  and/or  there  is 
a  limited  number  of  times  (perhaps  only  once)  that  a  system  can  be  tested. 

For  large  solid  rocket  engines,  each  of  these  situations  occur.  The  specific 
details  of  the  proposed  solutions  are  presented  after  the  practical  and  statistical 
problems  are  discussed.  It  should  be  noted  that  this  paper  specifies  how  the  reli¬ 
ability  of  the  system  can  be  evaluated;  it  does  not  discuss  how  reliability  may  be 
improved. 

Objectives  and  Problems 

The  objectives  are  to  define  and  measure  during  a  research  and  development 
program  characteristics  which  are  indicative  of  the  engines  ability  to  reliably  per¬ 
form  its  function  of  propelling  a  missile  of  one  or  more  stages  into  a  certain  tar¬ 
get  area.  To  accomplish  these  objectives  three  major  practical  and  statistical 
problems  must  first  be  considered: 

a.  Relating  engine  reliability  requirements  in  an  R  and  D  program  to  the 
weapon  system's  operational  reliability  requirements. 

b.  Obtaining  valid  reliability  data  in  an  R  and  D  program  when  the  objectives 
of  the  tests  vary  and  the  configuration  of  the  engine  undergoes  continous 
modification. 

c.  Making  efficient  reliability  estimates  based  upon  the  results  of  a 
limited  number  of  full-scale  engine  test  firings. 

Discussion  of  Problems 

a.  The  ultimate  intention  for  a  given  missile  is  that  it  will  fall  within  a 
certain  target  area.  This  intention  can  be  projected  into  single-stage 
engine  reliability  requirements  as  implying  that  for  successful  operation 
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of  the  missile  there  should  he  no  catastrophic  failure  vbicii  would  result 
in  mission  abort  nor  any  unsatisfactory  performance  which  would  result  in 
a  significant  target  miss.  On  the  other  hand,  operation  of  a  given  engine 
which  exhibits  no  catastrophic  failure,  and  for  which  the  performance  param¬ 
eters  all  lie  within  specified  limits,  would,  assuming  successful  operation 
of  the  remainder  of  the  missile,  produce  a  scatter  of  hits  in  and  around 
the  target  area.  The  ground  rules  which  interpret  the  success  or  failure 
criteria  of  the  weapon. system  into  the  success  or  failure  criteria  of  the 
engine  may,  to  a  certain  extent,  he  based  on  engineering  and  arbitrary 
Judgments.  However,  providing  the  ground  rtii.ee  are  clearly  stated,  under¬ 
stood,  and  rigorously  applied,  reliability  intimates  can  be  made  which  are 
valid  within  the  framework  of  those  ground  rules.  Furthermore,  only  develop¬ 
mental  test  stand  firings  can  be  utilized  for  reliability  evaluation  during 
the  program,  since  flight  tests  will  not  yet  have  been  conducted.  Thus,  the 
effect  of  performance  interactions  of  an  engine  with  the  remainder  of  the 
systems  in  the  missile  cannot  be  comprehensively  known  during  this  period. 
Therefore,  if  reliability  is  defined  as  the  probability  of  a  successful 
operation  of  the  weapon  system,  then  the  reliability  of  the  engine,  as  a 
functioning  subsystem  of  that  weapon  system  can  be  'nterpreted  as  the 
probability  of  successful  operation  of  the  engine.  This  in  turn  can  be 
regarded  as  the  probability  of  performing  within  engine  specifications. 

Thus,  the  relationship  between  the  weapon  system  and  engine  is  defined  by  the 
specifications . 

The  numerical  value  of  reliability  which  the  engine  may  be  expected  to 
display  or  contractually  demonstrate  would  depend  on  the  ’’state  of  the  art” 
of  the  engine,  the  number  of  units  available  for  testing,  and  the  reliability 
requirement  of  the  weapon  system. 

b.  !Hie  applicability  of  engineering  development  tests  for  reliability  evalua¬ 
tion  is  dependent  on  the  validity  of  the  data  and  the  intention  and  circum¬ 
stances  of  the  tests  generating  the  data.  Generally,  the  nature  of  develop¬ 
mental  testing  differs  from  reliability  testing.  Reliability  testing  would 
usually  involve  a  large  homogeneous  sample  of  engines  representative  of  the 
final  configuration.  It  is  generally  not  practical  to  produce  a  large 
number  of  expensive  engines  specifically  to  demonstrate  the  system’s  reli¬ 
ability.  Even  if  this  were  done,  by  the  time  production  and  testing  were 
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competed  the  reliability  results  would  refer  to  an  obsolescent  configura¬ 
tion,  since  simultaneous  engineering  testing  would  probably  have  resulted 
in  further  design  improvements .  Consequently,  in  this  program  reliability 
must  be  estimated  from  the  results  of  R  and  D  engineering  tests  -  However, 
engineering  testing  is  usually  performed  with  small  sample  sizes  on  changing 
configurations.  Therefore,  it  is  necessary  to  take  into  account  the  objec¬ 
tives  of  an  engineering  test,  to  ascertain  how  these  objectives  differ  from 
those  of  a  reliability  test  and  to  see  if  it  is  possible  to  reconcile  the 
two.  This  must  be  done  without  restricting  the  exploratory  nature  of 
development  testing. 

This  is  accomplished  by  "screening"  the  engineering  tests  for  reli¬ 
ability  use  by  means  of  the  "Declaration  Policy"  and  determining  the  degree 
of  representation  of  the  engine  towards  its  flight  configuration.  Thus, 
even  though  the  item  is  evolving  during  the  R  and  D  test  period,  representa¬ 
tive  data  can  be  obtained  and  used  to  determine  a  valid  estimate  of  reliability, 
c.  The  foregoing  discussion  implies  that  only  full-scale  engine  tests  will  be 
used  for  reliability  evaluation.  A  development  program  for  state-of-the- 
art  solid  propellant  engines  involves  a  great  deal  of  experimental  testing 
with  subscale  engines.  Initial  feasibility  studies  for  new  or  modified 
propellant  formulations  are  best  and  most  economically  undertaken  in  small 
and  subscale  engines  in  order  to  provide  evidence  that  the  propellant  will 
satisfy  Internal  ballistic  requirements  in  the  full-scale  engines.  In  many 
instances,  accurate  scaling  predictions  of  internal  ballistic  performance 
can  be  established.  Further  testing  with  subscale  engines  is  mdertaken  in 
order  to  determine  properties  of  charge  and  case  designs,  insulation  materials, 
movable  nozzle  designs,  etc.  for  the  full-scale  engine. 

However,  there  exist  differences  between  subscale  and  full-scale  engines 
which  cannot  be  completely  resolved  by  the  use  of  scaling  or  correlation, 
factors.  For  example,  these  differences  might  be  due  to  the  lack  of  suffi¬ 
cient  knowledge  of  the  mechanical  properties  of  scaled -up  propellant  charges, 
such  as  of  stress  magnitudes  and  patterns,  which  are  intimately  related  to 
problems  of  propellant  cracking  and  propellant-liner  separation. 
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Thus,  while  the  engineering  information  obtained  from  subscale  engine 
tests  is  essential  for  the  development  of  performance  and  reliability  of 
the  full-scale  engine,  the  best  evidence  to  ascertain  that  the  end  product 
requirements  will  be  met  is  obtainable  only  from  te,.u.e  of  full-scale  engines 
sufficiently  representative  of  final  design  configuration. 

Since  there  are  only  a  limited  number  of  full-ucale  engines  representa¬ 
tive  of  final  configuration  available  for  testing,  the  statistical  method 
of  making  reliability  evaluations  becomes  of  great  importance.  The  statis¬ 
tical  technique,  introduced  in  this  program,  supersedes  "the  product  rule" 
method  by  allowing  more  efficient  confidence  limits  to  be  estimated. 

Description  and  Conduct  of  the  Program 

The  program  is  divided  into  four  parts: 

a.  A  method  of  system  apportionment. 

b.  A  method  of  classifying  R  and  D  test  results  for  reliability  evaluation. 

c.  A  declaration  policy. 

d.  A  new  technique  of  estimating  reliability. 

System  Apportionment 

Tto  compensate  for  the  relatively  small  number  of  tests  of  full-scale  engines,  it 
is  essential  that  all  representative  data  be  utilized.  This  is  done  by  apportioning 
the  engine  into  three  Principal  Subsystems,  and  each  engine  test  is  evaluated  in  terms 
of  the  behavior  of  the  Principal  Subsystems.  Thus,  the  fact  that  an  engine  is  not 
fully  representative  of  the  final  configuration  in  any  one  test  will  not  prevent  the 
evaluation  oi'  those  Subsystems  which  cure  operating  in  a  configuration  cr  manner 
representative  of ilight  status.  The  three  Principal  Subsystems  are: 

a.  The  Propellant  Charge-Ignition  Subsystem. 

b.  The  Case-Liner-Internal  Insulation  Subsystem. 

c.  The  Thrust  Vector  Control  Subsystem. 

Engine  reliability  estimates  are  made  from  the  Principal  Subsystem  test  results 
(tested  within  the  environment  of  the  full-scale  engine)  and  can  begin  with  the  first 
test  firing. 

Applicability  of  Principal  Subsystem 

Hie  ftincipal  Subsystems  tested  in  a  full-scale  engine  firing  will  be  classified 
as  applicable  or  inapplicable  for  purposes  of  reliability  evaluation.  In  order  to 
determine  which  of  the  Subsystems  being  tested  in  any  full-scale  test  firing  are 
sufficiently  representative  of  flight  configuration  to  be  useful  for  reliability 
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evaluation,  it  is  necessary  to  set  up  criteria  which  determine  the  Subsystem's 
applicability. 


Principal  Subsystem 

i  —  ■  .  . . .  ■  —  -  ■■m.TJf.."-'  - -  1 

Required  Characteristics  for  Applicability 

Subsystem  A 

Propellant  Charge-Ignition 

W 

P 

=  propellant  charge  weight  lies  within  +1#  of 
model  specification  limits 

Subsystem  B 

Cas  e -Li ner - Internal 
Insulation 

1) 

Engine  configuration  includes  flight-weight 
case  and  end  closure  as  specified  by  current 
weight  and  balance  status  report. 

2) 

Propellant  weight  as  specified  for  Subsystem  A 

Subsystem  C 

I) 

Engine  configuration  includes  flight-weight 
movable  nozzles. 

2) 

Flight-weight  APS  actuator  subassembly. 

3) 

Propellant  weight  as  specified  for  Subsystem  A 

Thrust  Vector  Control 

Nozzles  must  be  intended  to  actuate  during 
test  firing. 

5) 

Predicted  action  time  not  less  than  model 
specification  limit  unless  thrust  termination 
operation  is  being  tested. 
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Test  Result  Classification 

Those  tests  which  are  applicable  will  be  evaluated  and  categorized  as  Exclusion, 
Failure  or  Success .  A  result  may  be  excluded  prior  to  this  test,  the  circumstances 
which  permit  this  are  listed  below: 


1) 


2) 


3) 


4) 


5) 


All  Principal  Subsystems  (A,  B,  C) 

The  Principal  Subsystem  does  not  have  required  characteristics 
for  applicability. 

Due  to  the  intention  or  circumstances  of  the  test,  a  particu¬ 
lar  Subsystem  may  be  excluded  due  to  stated  uncertain  charac¬ 
teristics  of  operation  relating  to  performance  and/or  possi¬ 
bility  of  malfunction;  e.g.,  if  inspection  shows  propellant 
voids  or  cracks  of  a  sufficient  degree  so  that  malfunction 
would  be  expected;  then  the  Propellant  Charge-Ignition  Sub¬ 
system  may  be  considered  for  exclusion. 

If  internal  ballistic  performance  is  predicted  to  be  outside 
current  Model  Specification  limits,  but  the  test  firing  is 
approved,  then  performance  will  be  excluded.  A  maximum  of 
two  exclusions  under  this  ground  rule  prior  to  the  formal 
FFRT  test  program  is  allowable.  No  exclusions  under  this 
ground  rule  are  allowable  during  the  formal  FFRT  program. 
Provisional  Exclusion:  A  Subsystem  will  be  excluded  if  there 
is  a  failure  of  an  experimental  part  which  has  been  so  listed 
on  the  Declaration  Form  prior  to  the  test,  and  which  is  being 
tested  for  the  first  time  in  a  full-scale  engine.  However, 
if  the  Subsystem  fails  due  to  the  failure  of  a  nonlisted  part, 
it  will  be  classified  as  a  failure. 

Provisional  Exclusion:  A  Subsystem  will  be  excluded  if  there 
is  a  failure  of  an  obsolete  part  which  has  been  so  listed  on 
the  Declaration  Form  prior  to  the  Test.  An  obsolete  part  is 
defined  as  a  part  used  in  a  test  configuration  for  reasons  of 
expediency,  but  for  which  there  already  exists  a  Reliability 
Design  Change*  for  that  part. 


*  A  Reliability  Design  Change  is  defined  as  a  modification  to  correct  a  previously 
observed  failure,  and  must  be  intended  to  appear  on  all  subsequent  engines. 

NOTE:  Reasons  for  all  exclusions  must  appear  on  the  Declaration  Form  prior  tc  the  tests. 
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Each  Principal  Subsystem  will  be  classified  as  having  succeeded  or  failed  depending 
on  whether  its  performance  in  operational  use  would  have  resulted  in  a  successful  or 
failing  flight.  The  exception  to  this  would  occur  when  for  causes  external  to  the 
Subsystem,  the  Subsystem  was  not  given  the  opportunity  to  succeed  or  fail.  The 
detailed  ground  rules  are  given  below: 


Propellant  Chaige-lgnition  I'Mbsystem  (A) 


1)  The  Subsystem  does  not  fail  and  is  not  excluded;  and  perform¬ 
ance  is  within  the  current  Model  Specification  limits. 

2)  It  operates  without  failure  with  performance  outside  current 
Model  Specification  limits,  provided  that-  thi.i  intention  is 

so  stated  on  the  Declaration  Form,  and  approved,  prior  to  test. 
(See  limitation  above) 


1)  The  igniter  fails  to  operate  or  fails  to  ignite  the  propellant. 

2)  The  ignition  delay  is  greater  than  the  maximum  valuer'  specified 
by  the  current  Model  Specification  limits. 

3)  The  ignition  peak  pressure/thrust  is  greater  than  the  equilib¬ 
rium  chamber  pressure/thrust. 

h)  Rough  combustion;  i.e.,  chamber  pressure/  thrust  peaks  —10$ 
of  equilibrium  chamber  pressure/thrust. 

5)  Engine  blow-up  attributable  to  propellant  or  igniter. 

6)  The  performance  values  lie  outside  the  current  Model  Specifi¬ 
cation  limits  when  the  performances  were  declared  to  be  within 
the  current  Model  Specification  limits. 


1)  Tests  which  fail  due  to  causes  external  to  Subsystem  A;  e.g., 
other  Principal  Subsystem  failures,  test  operator  error,  instru 
mentation  or  facility  malfunction,  provided  that  '"allure  of 
Subsystem  A  had  not  already  occurred. 

2)  Failure  of  the  Subsystem  due  to  the  failure  of  an  obsolete  or 
experimental  part  so  listed  prior  to  the  test  on  the  Declara¬ 
tion  Form. 

3)  Tests  in  which  Subsystem  A  did  not  fail  but  did  not  have  the 
opportunity  to  satisfy  the  declared  intention  of  the  test. 
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Ground  Rules  for  Classification  of  Test  Results  (Cont'd) 


Case-Liner -Internal  Insulation  Subsystem  (B) 

SUCCESS 

l)  The  Subsystem  does  not  fail  and  is  not  excluded. 

FAILURE 

l)  Case  or  end-closure  burn -through  during  normal  operation. 

POST-TEST 

EXCLUSION 

1)  Tests  which  fail  due  to  causes  external  to  Subsystem  B;  e.g., 
other  Principal  Subsystem  failures,  test  operator  error, 

instrumentation  or  facility  malfunction,  provided  that  failure 

of  Subsystem  B  had  not  already  occurred. 

2)  Failure  of  the  Subsystem  due  to  the  failure  of  an  obsolete 
or  experimental  part  so  listed  prior  to  the  test  on  the 

Declaration  Form. 

3)  Tests  in  which  Subsystem  B  did  not  fail  but  did  not  have 
the  opportunity  to  satisfy  the  declared  intention  of  the 

test. 
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_ 

Thrust  Vector  Control  Subsystem  (C) 

SUCCESS 

l)  The  Subsystem  does  not  fail,  is  not  excluded  and  performs  as 

programmed  for  the  duration  of  the  test. 

l)  Movable  nozzles  stick,  jam,  or  do  not  deflect  as  programmed. 

2)  Gas  leakage  or  burn-through  in  any  part  of  the  movable  nozzle 

e 

assembly  occurs. 

E 

* 

3)  Nozzles  under-  or  over -travel  intended  deflection  >  0.5  degrees 

< 

Pc 

4)  APS  does  not  deliver  rated  power  or  malfunctions. 

5 )  Actuators  malfunction. 

l)  Tests  which  fail  due  to  causes  external  to  Subsystem  C;  e.g.. 

s 

other  Principal  Subsystem  failures,  test  operator  error, 

H 

S3 

instrumentation  or  facility  malfunction,  provided  that  failure 

3 

O 

of  Subsystem  C  has  not  already  occurred. 

s 

2)  Failure  of  the  Subsystem  due  to  the  failure  of  an  obsolete  or 

E-< 

experimental  part  so  listed  prior  to  the  test  on  the  Declare- 

l 

EH 

tion  Form. 

CQ 

£ 

3)  Tests  in  which  Subsystem  C  did  not  fail  but  did  not  have  the 

_ 

opportunity  to  satisfy  the  declared  intention  of  the  test. 

NOTES:  It  may  be  required  that  the  engine  contractor  be  held  responsible  for 

attaining  a  numerical  reliability  requirement.  This  implies  that  the 
numerical  reliability  requirement  should  not  encompass:  (l)  equipment 
developed  by  another  associate  contractor  which  is  tested  in  conjunction 
with  the  engine  contractor's  system;  nor  (2)  interface  attachments,  the 
function  of  which  is  a  joint  responsibility  between  two  or  more  associate 
contractors.  When,  however,  equipment  is  furnished  bj,  vendor  or  sub¬ 
contractor  to  the  engine  contractor,  and  does  not  fall  in  category  (2), 
the  equipment  is  considered  to  be  the  engine  contractor's  responsibility 
with  respect  to  meeting  any  numerical  reliability  requirements. 
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Declaration  Pplicy 

In  order  to  establish  the  integrity  of  the  data,  it  h  necessary  to  determine 
the  intentions  of  a  test  and  by  applying  the  ground  rules  for  applicability  estab¬ 
lish  the  utility  of  the  test  results  for  reliability  purpose.  This  is  done  by 
means  of  the  Declaration  Form  which  must  be  completed  prior  to  each  full-scale  test 
as  part  of  the  Test  Plan.  It  is  then  submitted  to  the  program  manager  for  approval. 
The  form  is  self-explanatory,  but  it  is  appropriate  to  note  that  when  exclusions  are 
declared  the  reasons  should  be  given,  together  with  any  substantiating  information 
and/or  references  to  failure  reports,  inspection  reports,  engineering  change  orders, 
etc.  For  example,  if  a  part  is  provisionally  excluded  by  reasons  of  obsolescehce, 
the  engineering  change  order  number  or  the  new  part  (drawing)  number,  etc.,  should 
be  given  on  the  form.  (Figure  1  is  a  sample  Declaration  Form. )  The  Program  Manager 
might  establish  a  limit  to  the  number  of  provisioned  exclusions. 

Reliability  Reporting  and  Estimation 

The  Reliability  Report  Form  (Figure  2  is  a  sample  Reliability  Report  Form)  is 
completed  as  soon  as  possible  after  the  test  data  is  reduced.  The  data  are  then 
evaluated  by  the  contractor's  reliability  group  on  the  basis  of  the  ground  rules  and 
the  information  contained  on  the  Declaration  Form.  This  is  done  for  each  Principal 
Subsystem  after  each  full-scale  test.  A.  short  description  of  the  failure,  or  reason 
for  exclusion  when  any  Principal  Subsystem  is  so  classified,  should  appear  in  the 
remarks  column.  When  a  failure  occurs  which  cannot  be  assigned  to  any  particular 
subsystem,  this  fact  should  be  noted  as  well.  References  to  failure,  and  corrective 
action  reports,  etc.,  should  also  be  given  as  necessary.  The  Reliability  Report 
Forms  covering  a  particular  calendar  month  can  then  be  gathered  in  chronological 
order  and  summarized  as  illustrated  in  the  example  to  follow.  The  best  estimate  and 
95#  lower  confidence  limit  of  reliability  can  then  be  obtained  by  the  methods  given 
later  and  reported  monthly. 

Representative  and  Current  Data 

Because  of  the  small  number  of  tests  available  for  evaluation,  each  Principal 
Subsys+em  must  necessarily  display  a  relatively  high  reliability,  and  few  failures 
must,  therefore,  occur.  However,  at  the  beginning  of  the  program  several  failures 
may  occur,  establishing  such  a  high  cumulative  failure  rate  that,  were  there  no 
provision  for  discarding  data,  an  exorbitantly  large  number  of  successful  tests  would 
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be  needed  before  the  earlier  failure  rate  would  be  "absorbed."  This  is  not  feasible, 
and  in  a  development  program  which  expects  to  improve  the  product,  it  is  not  realistic 
to  handle  the  data  this  way.  If  these  failures  were  random  failures  or  failures  due 
to  unassignable  causes,  then  it  would  not  be  legitimate  to  discard  the  earlier  data, 
as  this  failure  pattern  would  be  the  manifestation  of  the  inherent  reliability  and, 
as  such,  would  indicate  that  the  Subsystem  was  not  sufficiently  reliable.  However, 
the  earlier  failures  are  not  generally  random;  i.e.,  they  do  have  assignable  causes, 
and  in  a  development  program  are  subject  to  analysis  and  corrective  action. 

This  corrective  action  is  called  a  Reliability  Design  Change,  which  is  defined 
as  a  modification  to  correct  a  previously  observed  failure  and  must  be  intended  to 
appear  on  all  subsequent  engines.  Data  generated  after  the  Reliability  Design 
Change  will  be  regarded  as  homogeneous,  and  earlier  data  discarded  as  being  no 
longer  representative  of  the  current  design.  Data  produced  sifter  the  Reliability 
Design  Change  will  be  called  Current  Data.  The  decision  as  to  what  constitutes  a 
Reliability  Design  Change  is  subject  to  the  approval  of  the  Program  Manager  and 
relates  only  to  that  particular  Subsystem  for  which  corrective  action  has  been  taken. 

All  engine  reliability  estimates  (as  described  in  the  example  below)  will  be 
based  on  Current  Data  only.  Thus,  when  counting  applicable  tests  the  count  should 
not  be  extended  any  further  back  than  the  last  Reliability  Design  Change  for  each 
Principal  Subsystem. 

Performance  Reproducibility 

Performance  reproducibility  is  not  incorporated  into  the  Reliability  evaluation 
method  described  in  this  paper;  i.  e.,  it  is  not  incorporated  into  the  reliability 
estimates  to  be  reported.  However,  generally,  Model  Specifications  contain  require¬ 
ments  relating  to  between  engines  performance  variation.  Applicable  data  obtained 
from  the  Reliability  Report  Form  would  be  used  to  determine  the  probability  of 
meeting  these  variability  requirements.  Further,  if  these  pertinent  performance 
parameters  are  suitably  chosen;  e.g.,  specific  impulse,  thrust,  weight,  etc.,  the 
probability  of  the  missile  falling  within  the  C.E.P.  can  be  computed  on  a  propulsion 
reliabilicy  basis. 
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Engine  Reliability  Estimation 
Technique  of  Estimation* 

Engine  reliability  will  be  computed  from  Principal  Subsystem  test  results. 

After  each  test,  it  will  generally  be  possible  to  classify  each  of  the  Principal 
Subsystems  as  having  succeeded,  failed,  or  been  excluded.  The  exception  to  this 
classification  can  occur  when  there  is  a  failure  but  it  is  not  known  which  Principal 
Subsystem(s)  failed.  While  it  is  desirable  to  be  able  to  classify  completely  each 
test,  an  unassignable  failure  still  represents  a  system  failure  and  must  be  incor¬ 
porated  into  any  system  reliability  estimate.  The  general  procedure  is  as  follows: 

The  number  of  applicable  tests  on  each  Principal  Subsystem  are  determined.  The 
minimum  of  these  three  numbers,  N,  gives  the  number  of  equivalent  engine  tests  per¬ 
formed.  To  compute  the  reliability  for  the  engine  it  is  necessary  to  count  the  number 
of  known  failures  which  occurred  in  the  last  N  tests  of  each  Subsystem.  In  addition, 
those  failures  which  have  not  been  assigned  or  attributed  to  any  Subsystem  in 
particular  are  arbitrarily  assigned  to  the  individual  Principal  Subsystems.  This  is 
done  for  all  possible  arrangements  consistent  with  the  number  of  failures  and  tests 
appearing  in  the  unassignable  category.  Thus-  when  the  known  failures  are  added  to 
the  arrangements,  a  set  of  all  possible  failure  arrangements  for  a  given  number  of 
equivalent  engine  tests  is  obtained.  Any  one  failure  arrangement  can  be  written 
(N;  f^,  fg,  f^),  where  f^,  fg>  f^  are  the  number  of  failures  of  the  separate  Sub¬ 
systems;  and  f^  +  f g  +  f^  is  therefu*e  the  total  number  of  assignable  and  unassign¬ 
able  failures  during  the  period  of  .T  eau?  alent  engine  tests.  The  minimum  reliability 
for  95  percent  confidence  can  then  be  . btained  from  the  tables or  graph  for  each 
(N;  f^,  fg,  f^).  Some  of  the  numerical  values  of  these  reliabilities  will  be  the 
same  since  the  value  of  the  reliability  is  independent  of  the  order  of  f^,  fg  and  f^. 

The  smallest  of  the  reliabilities  is  taken  as  the  minimum  demonstrated  reliability 
which  could  occur  in  a  population  represented  by  the  sample.  If  later  analysis  per¬ 
mits  unassignable  failures  to  be  allocated,  the  estimates  will  then  be  recomputed. 

The  best  estimate  of  reliability  for  the  same  sample,  is  obtained  by  taking  that 
combination  of  failures  which  gave  the  smallest  of  the  reliabilities  for  the  confidence 
estimate  and  substituting  in  (if  -  f , )  (N  -  f?)  (N  -  f0)/N3. 

A  J 

* 

The  mathematical  basis  for  the  method  of  reliability  estimation  is  based  upon  the 
work  described  in  References  (l)  an.:.  (2)  and  the  tables  in  Reference  (3) • 
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An  Example  of  Estimation 

a.  An  example  will  illustrate  the  method  of  engine  reliability  estimation. 

The  method  of  tabulating  the  results  as  shown  in  the  table  below  will  be  found  conven¬ 
ient.  During  each  test  all  Principal  Subsystems  are  physically  present  but  may  or 
may  not  be  applicable  for  reliability  evaluation  according  to  the  ground  rules.  The 
result  of  the  test  for  each  of  Subsystems  A,  B,  and  C  will  be  success  (S),  failure  (F) 
or  exclusion  (which  is  indicated  by  a  blank  in  the  table).  In  addition,  there  may 
be  unassignable  failures.  These  are  indicated  under  that  combination  of  Subsystems 
(AB,  BC,  CA  or  ABC)  which  contains  the  potential  failing  Subsystem(s).  The  following 
table  of  hypothetical  results,  together  with  accompanying  detailed  explanation,  will 
clarify  the  above  discussion. 


Test  No. 

Reliability 

Report 

No. 

Date  of 

Assignable 

Results 

* 

Unassignable  Failures 

(Chrono¬ 

logical) 

Test 

A 

B 

c 

wr 

ABC 

1 

S 

2 

F 

3 

S 

s* 

b 

F 

5 

s* 

s* 

F* 

6 

s* 

F* 

7 

S* 

s* 

S* 

8 

F* 

9 

S* 

s* 

i 

i 

1 

No.  of 

Subsystem  Times  Tested  Assignable  Failures  Possible  Failures 


A  8  0 

B  7  1 

C  5  1 


2 

3 

2 


*  See  paragraph  b,  following  page. 
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Test  No. 
1 

2 

3 

4 


5 

6 

7 

8 

9 


Explanation  of  Tabulation 

Subsystem  A  was  applicable  and  a  success . 

B,  C  were  excluded. 

Subsystem  B  was  applicable  and  a  failure. 

A,  C  were  excluded. 

Subsystems  A  and  C  were  applicable  and  successes. 

B  was  excluded. 

Subsystems  A  and  B  were  both  applicable  but  a 
failure  occurred;  post-test  analysis  failed  to 
establish  which  of  the  two  Subsystems  failed  or 
whether  it  was  both.  C  was  excluded. 

A,  B  were  both  successful;  C  a  failure. 

A  was  a  success  but  an  unassignable  failure  occurred 
in  B  and/or  C. 

All  Subsystems  successful. 

All  Subsystems  applicable  but  (an)  unassignable 
failure(s)  occurred  in  one  or  more  of  them. 

A  and  B  were  both  successful.  C  was  excluded. 


b.  In  the  above  example,  the  Principal  Subsystem  experiencing  the  fewest 
number  of  tests  is  C  with  5;  thus,  a  maximum  of  the  equivalent  of  5  complete  engines 
have  been  tested.  Counting  back  from  Test  No.  9  for  the  last  5  tests  of  each  Princi¬ 
pal  Subsystem,  it  is  found  that  A  was  present  and  applicable  in  Tests  No.  9,  8,  7, 

6,  5i  Subsystem  B  in  Tests  No.  9>  8,  7t  6,  5  and  Subsystem  C  in  Tests  No.  5,  7,  6, 

5  and  3*  Thus,  only  the  results  of  the  aforementioned  tests  for  each  Subsystem 
should  be  utilized  in  obtaining  the  engine  reliability  estimate,  i.e.,  only  those 
results  marked  by  an  asterisk  in  the  table.  During  the  period  of  testing  the  5 
engines,  A  has  no  known  failures  and  1  possible  failure  in  Test  No.  8.  B  has  no 
known  failures,  2  possible  failures  (one  in  Test  No.  6  and  one  in  Test  No.  8).  C 
has  one  known  failure  in  Test  No.  5  and  2  possible  failures  in  Tests  No.  6  and  8. 

£11  the  possible  arrangements  of  the  unassignable  failures  are  shown  below. 
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Subsystem  A 

Subsystem  B 

Subsystem  C 

Total  Unassignable  Failures 

0 

0 

2 

2 

0 

1 

1 

2 

0 

2 

0 

2 

1 

0 

1 

2 

1 

1 

0 

2 

The  known  failures:  0,  0,  1,  respectively,  are  added  to  each  of  the  above 
arrangements  to  give  all  possible  variations,  both  known  and  unassignable;  in  the 
notation  (N;  f 2,  f^)  these  become 

(5;  0,  0,  3) 

(5;  o,  l,  2) 

(5;  o,  2,  l) 

(5;  l,  o,  2) 

(5;  l,  l,  i) 


From  Figure  3  it  is  seen  that  the  first  combination  gives  a  value  of  7-6  per 
cent  for  minimum  demonstrated  reliability  at  the  95  per  cent  confidence  level.  The 
next  three  combinations  give  16.3  per  cent  and  the  last  combination  gives  18.9  per 
cent. 


The  minimum  of  the  possible  reliabilities,  i.e.,  7*6  per  cent  with  95  per  cent 
confidence  is  the  reported  reliability.  In  this  example,  based  on  hypothetical  test 
results,  the  possible  reliabilities  vary  considerably  because  of  two  facts.  Firstly, 
there  is  a  much  greater  proportion  of  unassignable  failures  than  known  failures  and, 
therefore,  the  arrangements  greatly  affect  the  reliability  estimates,  and  secondly, 
the  sample  size  is  very  small.  Generally,  this  will  not  be  the  case,  so  that  the 
reliability  estimates  will  be  much  closer  together. 

The  best  estimate  of  reliability,  as  obtained  from  the  worst  combination  of 
failures  (0,  0,  3)>  given  by  the  formula  is 

(5  -  0)  (5  -  0)  (5  -  3)/53  =  2/5  =  bofi 

Use  of  Figures  3  and  Table  I _ 


TiM  Q  Cfi  V^c  f  V10  6cf -Imn+n  -I  aVk’i  T  ■?  tv  0+  0  QQ  r\n>*  npnf  1  oirol 

A  A  r>«*  A  w  J  Q*  *  Wto*  Vfc/  *—  *  ‘  *  ...  v  A  A  w  A  A  VAW  AAA  W  j  WAV  V  W»i  W  W  V  AAA  A  \*  VltV  W  A  W  »  VA 

for  a  given  number  of  trials,  N  =  5  to  Uo,  and  a  particular  combination  of  failures 
(f^,  fp,  f^)*  The  figure  covers  values  of  (f^  +  f^  +  f  )  from  0  to  20;  however,  not 


Page  17 


all  failure  combinations  are  plotted.  Table  I  lists  in  descending  order  with  respect 

to  the  reliability  values,  various  failure  combinations.  Only  these  combinations 

marked  with  a  single  asterisk  have  been  plotted.  Thus,  if  it  is  necessary  to  obtain 

a  reliability  estimate  for  a  combination  not  plotted,  e.g.,  (3>  3>  1)>  it  can  be  done 

for  any  particular  N  by  interpolating  between  the  curves  for  (3,  2,  2)  and  (4,  2,  l).  Ref 

erence  3  allows  reliability  estimates  to  be  obtained  for  a  wider  range  of  N  and  various 
confidence  levels. 

Conclusion 

The  paper  has  shown  how  the  reliability  of  an  engine  can  be  estimated  from 
engineering  development  tests  without  restricting  the  exploratory  nature  of  the  tests. 
These  estimates  can  be  obtained  even  though  the  configuration  and/or  hardware  being 
evaluated  has  not  reached  its  final  design.  The  statistical  technique  used  in  the 
analsysis  of  the  data  results  in  a  more  efficient  confidence  estimate  and  thereby 
supersedes  the  product  rule  for  interval  estimation. 
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DECLARATION  FORM 


A. 


B. 


Engine  Serial  No._ 
Test  No. 


Engine  type 


Test  Stand 


Date  of  Test 


Applicable 


Not  Applicable 


Which  of  the  following  Principal  Subsystems  are  applicable  for  reliability 
evaluation? 

(1)  Propellant  Charge-Ignition  (A) 

(2)  Case-Liner-Internal  Insulation  (B) 

(3)  Birust  Vector  (C) 


If  any  of  the  Principal  Subsystems  are  not  applicable  state  reasons. 


D.  Which  performance  parameters  should  be  excluded  from  reliability  evaluation? 
State  reasons: 


E.  Which  components  or  parts  are  declared  experimental  or  obsolete  (for  provisional 
exclusion) ? _ 


F.  Consents: 


G.  Movable  Nozzle  Program:  Number  of  cycles,  angles  of  deflection,  and  period  of 
operation  _ 


Signatures  and  Approvals : 

Program  Management  _ _  Test  Engineer 

Date _ _  Reliability 


FIGURE  1 


2 


RELIABILITY  REPORT  FORM 


*;i_URE  COMBINATIONS 
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95  PER  CENT  LOWER  CONFIDENCE  LIMIT  TO  TRUE  RELIABILITY  FOR 
OBSERVED  FAILURE  COMBINATIONS  OF  A  THREE-SUBSYSTEM  SERIAL 
SYSTEM  WITH  N(5 SN<40)  TRAILS  PER  SUBSYSTEM 
(ALL  PERMUTATIONS  OF  FAILURE  COMBINATIONS  (f  ,  ,f2,  f  3)  ARE  EQUIVALENT.) 


I'TiiijRK  i 


NUMBER  OF  TRIALS 


Table  1.  List  of  Failure  Combinations  in  Order  of  Descending  Reliability 


Total  No.  Combination  Total  No.  Combination  Total  No.  Combination 
of  Failures  of  Failures  of  Failures  of  Failures  of  Failures  of  Failures 


+J2+f3) 

(£,. 

£2' 

V 

^1  +  f2  +f3^ 

(£,. 

f2’ 

V 

(f  1  +  f  2  +  f3> 

(£r 

*2’ 

V 

0 

0 

0 

0* 

8 

3 

3 

2* 

1 1** 

10 

0 

0* 

2 

2 

1 

1 

0 

0* 

4 

3 

1 

12** 

11 

0 

0* 

4 

4 

0 

2 

1 

1 

0* 

5 

2 

1 

13** 

12 

0 

0* 

2 

0 

0* 

5 

3 

0* 

6 

1 

1 

1 4** 

13 

0 

0* 

3 

1 

1 

1* 

6 

2 

0 

2 

1 

0* 

7 

l 

0* 

1<S** 

14 

0 

0* 

3 

0 

0* 

8 

0 

0* 

16** 

15 

0 

0* 

4 

2 

1 

1* 

9 

3 

3 

3* 

2 

2 

0* 

4 

3 

2 

1 7** 

16 

0 

0* 

3 

1 

0* 

4 

4 

1 

4 

0 

o* 

5 

2 

2 

18** 

17 

0 

0* 

5 

3 

1 

5 

2 

2 

1* 

5 

4 

0 

19** 

18 

0 

0* 

3 

1 

1* 

6 

2 

1 

3 

2 

1* 

6 

3 

0* 

20** 

19 

0 

0* 

4 

1 

0* 

7 

1 

1 

5 

0 

0* 

7 

2 

0 

21** 

20 

0 

0* 

S 

1 

0* 

6 

2 

2 

2* 

9 

0 

o* 

3 

2 

1* 

3 

3 

0 

10 

4 

3 

3* 

4 

1 

1 

4 

4 

2 

4 

2 

0* 

5 

3 

2 

5 

1 

0* 

5 

4 

1 

6 

0 

0* 

5 

5 

0 

6 

2 

2 

NOTE: 

7 

3 

2 

2* 

6 

3 

1 

3 

3 

1 

6 

4 

0 

All  permutations  of 

4 

2 

1* 

7 

2 

1 

£1.  fz. 

are 

4 

3 

0 

7 

3 

0 

equivalent. 

5 

1 

1 

8 

1 

1 

5 

2 

0 

8 

2 

0 

6 

1 

0* 

9 

1 

0* 

7 

0 

0* 

10 

0 

0* 

Plotted  in  Figure  3. 

For  those  values  of  f  j  +  fj  ill  the  estimate  is  obtained  from 

IN;  (fj  +  fz  +  *3  —  0.  0,  0]  unless  two  of  fj,  {fr  fj  [li,  f3»  say)  are  zero,  in 
which  case  the  reliability  is  given  by  (N;  fj,  0,  0). 


